Locking in Returns: Speeding Up Q-Learning by Scaling

One problem common to many reinforcement learning algorithms is their need for large amounts of training, resulting in a variety of methods for speeding up these algorithms. We propose a novel method that is remarkable both for its simplicity and its utility in speeding up Q-learning. It operates by scaling the values in the Q-table after limited, typically small, amounts of learning. Empirical results in a variety of domains, including a partially observable multi-agent domain that is exceptionally difficult to solve using standard reinforcement learning algorithms, show significant speedups in learning when using scaling.